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Inventor(s) : David Cohen 

Title of the Invention 

Situation Monitoring in Virtual Reality 

Field of the Invention 

The present invention relates to 3ituation monitoring usin. 

virtual reality " 


Background of the Invention 

It is desirable to know when personnel encounter emergency sitaaUons 
In particular security personnel including night watchman and guards, airUne 
Pilots, truck and van drivers and the like can be the subject of attacks and other 
emergencies wia. which they are unable to cope. In such a case it is desirable 
for the subject of the attack to call for help, but somejtoes the nature of the 
emergency renders callmg for help hnpossible. Like^rse'. elderly and other 
vutaerable persons, particularly those livmg on their own. can fmd themselves 

m dtfaculties and unable to reach a telephone to call for help, for example after 
a fall. • 

In cases where it is not possible to call for help, a number of systems 
exist for automatically determining that an emergency situation exists and 
calhng for help, 

Hospital-based systems that monitor a patient's pulse and call a doctor 
or nurse if the pulse falls are well known but are not suitable for anything other 
than the hospital environment. 

Aircraft based hijack warning systems rely upon the pilot's standard 
radio-based voice link to air traffic control or include panic buttons for 
broadcasting an SOS-signal. Hijackers however tend to be familiar with the 
presence of these systems and either use them to their advantage or prevent 
their use altogether. 
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Other systems for protecting aircraft firom emergencies tend to rely on 
pilots' reaction times. Certain types of emergencies happen too quickly for the 
pilots to be able to raise the alarm or divert the pilots to emergency activity 
without diverting their attention to raising the alarm. 
5 Often, the ability to determine what has happened following an aviation 

disaster is dependent on finding the aircraft flight recorder or black box. 

Israel Patent Application No. 145498 to the present applicant discloses a. 
system for detecting cockpit emergencies comprising the following: 

a) an input unit for receiving body stress level information from at least two 
10 subjects, 

b) a detection unit, associated with said input unit, for comparing stress level 
information from said at least two subjects, to detect substantially simultaneous 
stress level increases in said subjects, 

the system being operable to threshold detected simultaneous stress level 
15 increases to infer the presence of an emergency situation and to enter an alarm 
state. 

The system uses the physiological state' of the pilots to lietermine that an 
emergency situation has arisen. In order to reduce false alarms it takes data 
from the two pilots and deduces the presence of an alarm when both pilots 

20 indicate stress. Such a system has the disadvantage that it is only useful in 
situations such as the cockpit of a civil aircraft where two or more persons are 
likely to undergo the same emergency. The system is not applicable to security 
guards, elderly people living alone and the like. Likewise it is not applicable 
for monitoring of persons being sent into dangerous situations such as troops 

25 into battle or firemen into a burning building. 

Another situation in which monitoring of the physiological and motion 
state of a human subject is useful is in virtual reality applications such as 
virtual reality games in which a subject's action is followed in real time on a 
computer screen or in the use of actors to create animated characters for 

30 animated films or games. Generally these applications are achieved by 

monitoring a large number of points on the subject's body and incorporating 
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them into a triangulated 3D model which reproduces the subject's movement 
with great accuracy. Unfortunately this produces a very large amount of data 
and requires a very large amount of processing. It also requires complex and 
specialized equipment for monitoring the movement of all of the points on the 
s,ubject's body in three dimensions. 

Summary of the Invention 

According to the present invention there is provided emergency situation 
detection apparatus comprising: 

a stress input unit for receiving body stress level infomiation from a 
subjects, 

a physical input unit for receiving body physical reaction data from said 
subject, 

a comparator unit, associated with said stress input unit and said 
physical mput unit, for comparing stress level information and physical 
reaction data, to detect substantiaUy simultaneous stress level increases and a 
physical reaction in said subject, ' ' ' ► . . 

said apparatus being operable to threshold said simultaneous detection to 
infer the presence of an emergency situation and to enter an alarm state. 


Brief Description of the Drawings 

For a better understanding of the invention and to show how the same 
may be carried into effect, reference wiU now be made, purely by way of 
example, to the accompanying drawings. 

With specific reference now to the drawings in detail, it is stressed tliat 
the particulars shown are by way of example and for purposes of illustrative 
discussion of the preferred embodiments of the present invention only, and are 
presented in the cause of providing what is believed to be the most useful and 
readily understood description of the principles and conceptual aspects of the 
mvention. In this regard, no attempt is made to show structural details of the 


invention in more detail than is necessary for a fundamental understanding of 
the invention, the description taken with the drawings making apparent to those 
skilled in the art how the several forms of the invention may be embodied in 
practice. In the accompanying drawings, 
5 Fig. 1 is a simplified diagram of a detection device according to a first 

embodiment of the present invention; 

Fig. 2 is a simplified diagram showuig the detection device of Fig. 1 in 
greater detail; and 

Fig. 3 is a simplified diagram showing a 3d virtual figure for providing 
10 an intuitive user firont end for monitoring the state of a subject or altematively 
for providing a way of translating motion of the subject into an animation for a 
virtual reality game or an animated film or the like. 

Description of the Preferred Embodiments 

15 The present embodiments provide a physiological situation and 3D 

motion detector which uses physiological responses of subjects such as the 
heart rate to measure mood or determine that an emergency siliiation exists and 
to automatically raise an alarm. An additional signal is taken firom an 
independent motion detector which measures three-dimensional motion of the 

20 body, perhaps using a position detector or an accelerometer or both. 

In an embodiment, additional motion detectors are added to any of the 
head, arms and legs as desired. The motion detectors are preferably passive 
signal sources whose positions are detected by triangulation firom two points on 
the main device. 

25 For emergency situation detection the signals firom the stress and the 

motion detector are monitored together to obtain an indication that such a 
situation has arisen. The use of both signals provides protection against false 
alarms say caused by self-induced anger, pure firight unaccompanied by an 
attack, and the like to- which individual subjects may be susceptible. The 

30 signals may be measured against a threshold, or a delta may be used. 
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In a broader sense the present embodiments provide indications of 
dangerous situations arising or of circumstances that could lead to dangerous 
situations. For example, the embodiments may be able to from physiological 
measurements that a security guard has fallen asleep, and therefore is not doing 
his duty of guarding. 

When used for virtual reality, the main interest is the position or motion 
detection. However the physiological detection can provide a mood indication. 

In the preferred embodiment the signals are translated into the motion of 
a three-dimensional animated figure on the screen. The figure uses the signals 
to replicate the mood and the motion of the subject. Thus in emergency 
situation detection a monitoring party has an intuitive user friendly indication 
of the state of persons he is monitoring, and in animation and games the 
anhnated character moves in accordance with the motions of the actor or player 
but without needing expensive detection equipment or large scale processmg 
15 ability. 

Before explaining at least one embodiment of the invention in detail, it 
is to be understood that the invention'is not liiiited kfite ippfic^^^^ to the ' 
details of construction and the arrangement of the components set forth in the 
following description or illustrated in the drawings. The invention is appUcable 
to other embodiments or of being practiced or carried out in various ways. 
Also, it is to be understood that the phraseology and terminology employed 
herein is for the purpose of description and should not be regarded as limiting. 

Reference is now made to Fig. 1, which shows an emergency situation 
detection apparatus placed on a user. 

In Fig. 1, a subject 10 has an emergency situation detector 12 attached 
thereto. The detector 12 comprises bodily function detector 14 and physical 
reaction detector 15. The bodily function detector may for example detect 
pulse rate or sweat levels of the subject. Preferably the detector may be 
concealed beneath the subject's clothing. The detector 12 is preferably able to 
send signals in non-contact manner to emergency situation detection apparatus 
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16. 


5 


30 


The bodily function detector 14 receives physiological body stress level 
mformation. The physical reaction detector preferably detects sudden 
movements, or indications of an impact of some kind or the attainment of a 
honzontal position or Ulce indicators of physical r^on. An impact detector 
mayforexamplecompriseapiezoelectricsensor. Neither physiological stress 
alone norphysical reaction alone ate reliable mdicators of an emergency 
situation but both taken together may be expected to mcrease the reliability of 
any determination of an emergency situation. 

^*f««"«i-"<'»"»»<i= to Fig. 2. which shows the detector 12 in greater 
detau. Preferably a comparison unit 20 is associated with the bodUy function 
and reaction detectors, for comparing signal mfotmation to determine 
substantiaUy simultaneous occurrence of stress level increase and physical 
reaction in the subject. The detections may be thresholded as deemed sensible 
by the stalled person to mcrease reliabiUty of detection. Different thresholds 
may be appropriate for different kinds of subject Thus elderly people may be 
better served by a lower threshold than a security guard. As a further 
alternative, mstead of a fixed ftreshold lev^l, ie sy^' iay monitor the 

in signal level over time. Tie change or delta may then be thresholded' 
TT^sholdmg the deltas can distinguish high signal levels which are due to a 

rapidly occurring event firom high signal levels which may be due to 
background stress and the like. 

The thresholded oxttpat of the comparison unit, foUowing a positive 
result of the thresholding, is passed to an alarm state manager 28 to imply the 
presence of an emergency situation and to enter an alarm state. 

Preferably fkc alarm state manager is able to call for assistance for 
example via automatic opening of a radio link, or of a Wdeo link, to a central 
controller, thus to provide hmnediate mdication of an emergency state 
Preferably, the link, which is at least an audio Unk, mcludes at one end a 
speaker and or microphone located on the body of the user. 

In a further prefenx»l embodiment specifically for an aircraft cockpit the 
almn state manager is able to mitiate an automatic download of the aircraft's 
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flight recorder or black box data to a central controller, thus making available 
flight information even if the black box is never recovered. 

The alarm state manager is preferably also able to enter an alarm state 
under the influence of other detectors, for example with detection of a loud 
noise or following prolonged instability. The alarm state manager may be able 
to enter different levels of alarm states prompting different actions. 

In a further preferred embodiment of the present invention, the 
emergency situation detector includes an audio or other confirmation channel 
which can be opened upon detection of an emergency in order to provide 
confutation of the situation or aUow two-way communication, or the like. 

In a further preferred embodiment the emergency situation detector 12 
includes a GPS detector to provide positioning information. For use in a 
building or other places where GPS signals may not be available, a 
triangulation system may be installed for accurate positional information. 

A further preferred embodiment intended for a user who stays within a 
predefmed area, such as a security guard on patrol, simply sends regular code 
signals from which the system infers th'at he is in jp^sifioii: ' " 

Further preferred embodiments are provided to determine attitude, 
position and motion of a subject. Thus the emergency situation detector may 
20 include an accelerometer. A detector for detection of a direction that a user is 
facing may be strapped to the chest or a like part of the body. The detector 
includes a compass needle and the relative alignment of the compass needle 
relative to a predefined forward direction of the body provides information as 
to the direction the user is facing. 

In an embodiment, additional motion detectors 16 are added to any of 
the head, arms and legs as desired. The motion detectors are preferably passive 
signal sources whose positions are detected by triangulation from two points on 
a main device 18. It is noted that main device 18 may replace detector 12. 
With the additional motion detectors it is possible to determine explicitly that 
the subject is running or walking or lying down or the like. 
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In a further preferred embodiment, emergency situation detectors are 
provided to two or more persons in a team. The signals from different 
members of the team can be compared to deteimine who is the closest to an 
event. For example the intensity of an audio signal as received from two 
different users can be compared to deteimine who was the closest to an 
explosion. The team can then be instructed accordingly to deal with the 
situation. 

In one embodiment, data is stored for a predetermined time in a stack 
forexampleaFIFOstaolc. The si^e of tl>e sUck may be a given amount of ' 
data, or may be a given amount of time, or some other fector as preferred In 
the event of the detection of an emergency situation, all of the data c^rently in 
the stack is saved, so as to aUow subsequent analysis, lUe stack embodiment is 
useful because it makes available information fiom directly before the 
emergency, often extremely usefiil in any investigation. 

^'""^^"'^offtepreseMmventionmayuseaprivatecommumcation 
channel. Other embodiments may make use of existing chamiels such as the 

cellular network. Yet other embodimetits day cotijitij^dnivirsai 
communicators which make use of pubUc netwotks if detected and use their 
own channel of communication otherwise. 

According to a firther embodiment a system comprises rule based logic 
and one or more body sensors for location on the subject. The subject is 
expected to follow certain behavioral rules, for example a guard patrols by 
walking around wiflun a certain area. If he were to run or lie down i, would be 
apparent that an abnormal situation may have arisen. Thus the sensor is usable 
m combination with the rule based logic to detect non-compliance with the 
behavioral rules, to indicate an abnormal situation and if necessary to set offan 
alarm or otherwise summon help. It will be clear that the more independent 
sensors are used die more reUable the determination can be. 

In other circumstances, a guard may be e,q,ected to run and lie down to 
observe suspicious citcumstances. In such a case the system may not react 
under such circ^nstances. but may await an additional indication such as an 
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impact or the sound of an explosion, or signs or rolling or the like, which may 
indicate that the guard is under attack. 

In a preferred embodiment, the detectors are programmable. The rules 
can be. changed for different users or for allowing the same device to given to 
different users having different requirements. The device can also be 
dynamically programmable according to parameters it is able to detect. Thus it 
may be able to use detected locations to change between different sets of rules. 
Or as another example, a device programmed for use by a fireman may change 
the rules it is using depending on the temperature it detects. In a further 
example the change of rules may be carried out on line, for example over a 
radio connection. 

A position or location detector may be used in combination with the 
above system and the rules preferably define location-based behaviors. 

For emergency situation detection the signals firom the stress and the 
motion detector are monitored together to obtain an indication that such a 
situation has arisen, as described above. The use of both signals provides 
protection against felse alarms say caiised By self-hi^S^d an^er, jjUre fright 
unaccompanied by an attack, and the like to which individual subjects may be 
susceptible. The signals may be measured against a threshold, or a delta may 
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When used for virtual reality, the main interest is the^ position or motion 
detection. However the physiological detection can provide a mood indication. 

In the preferred embodiment the signals are translated into the motion of 
a three-dimensional animated figure on the screen. Reference is now made to 
Fig. 3 which shows figure 30 that uses the signals to replicate the mood and the 
motion of the subject. The figure is a simpUfied figure and may be a 
preconfigured animation. 

Thus in emergency situation detection a monitoring party has an 
intuitive user fiiendly indication of the state of persons he is monitoring, and in 
animation and games the animated character moves in accordance with the 
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motions of the actor or player but without needing expensive detection 
equipment or large scale processing ability. 

If the subject being monitored is himself watching the screen then the 
animated character provides an interactive feedback to the subject. It is 
possible to carry out recreational and physical excercises and training with 
immediate and personalized feedback. A subject can be warned if he is not 
carrying out an exercise correctly or is carrying it out in a dangerous mamier. 
This may be determined by comparing the movement with a predetermined 
program of movements or comparing the movements with a feature on the 
virtual reality screen. Furthermore movement of a ball or the direction of a 
virtual gun can be monitored in combination with the movement of the subject 
to decide whether a participant in a game has scored points or is killed or the 


like. 


The system can monitor for sounds and the like for emotional content, 
for example laughter, crying and the like. Likewise the system can monitor tiie 
physiological signals for emotional cues. The emotional cues are then 
transferred to the animated figure. " " - ^^'-^'^ • • . 

As a further example if something happens to a subject being monitored, 

such as him being attacked, the animated figure clearly shows what is 
happening or what has happened to him at a particular time, since the 
information can be stored and replayed. 

It is appreciated that certain features of the invention, which are, for 
clarity, described in the context of separate embodiments, may also be provided 
in combination in a single embodiment. Conversely, various features of the 
invention which are, for brevity, described in the context of a single 
embodunent, may also be provided separately or in any suitable 
subcombination. 

It will be appreciated by persons skiUed in the art that the present 
invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defmed by the 
appended claims and includes both combinations and subcombinations of the 
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various features described hereinabove as well as variations and modifications 
thereof which would occur to persons skilled in the art upon reading the 
foregoing description. 
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Claims 

1 . Device for virtual reality representation of a subject being 
monitored, the apparatus comprising: 

a measurement part comprising: 

a physiological data input unit for receiving physiological 
information jfrom said subject, 

a motion unit for receiving body motion data from said subject, 
a comparator unit, associated with said physiological data input 
unit and said motion unit, for comparing physiological information and 
physical reaction data, to determine an overall state of said subject, and 
a monitoring unit for using signals received from said measurement part to 
animate a virtual animation character at a remote location, thereby to provide 
an indication of a current state of said subject. 

2. The device of claun I , wherein at least said physiological input 
unit and said motion unit are configured for attachment to said subject. 

3 . The device of claim 2, wherein said comparator unit is configured 
for attachment to said subject. 

4. The device of claim 3, wherein said motion uiiit comprises 
passive position detectors connected to a pluraUty of locations on said subject 
and a triangulation unit for detecting relative motions of said passive position 
detectors. , 


5. The device of claim 1, wherein said stress level information 
comprises pulse rate information. 

6. The device of claim 1 , wherein said stress level information 
comprises breathing rate information. 
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7. The device of claim 1 wherein said stress level information i 
sweat level information. 


8. The device of claim 1, adapted to use said physiological data to 
apply an emotional state to said animated figure. 

9. The device of claim 1 , configured to interpret extremely low 
stress level information as said subject being in a state of sleep. . 

10. The device of claim 1, wherein said motion unit further 
comprises an inclination detector afBbced to said subject. 


U . The device of claim 1 , wherein said motion unit comprises an 
accelerometer. 


12. The device of claim 1, wherein said measurement unit is 
respomive to trammitter units placed on said subjects? 

13. The device of claim 1, wherein said measurement unit is operable 
to enter an alarm state under predetermined conditions and said alarm state 
comprises automatic opening of a communication channel to a central 
controller. 

14. The device of claim 1, wherein said predetermined conditions 
include a position likely to cause injury. 

1 5. The device of claim 13, wherein said alarm state comprises 
automatic opening of an audio channel to a central controller. 

16. The device of claim 15, wherein an end of said audio channel is 
located on said subject 
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17.. The device of claim 13, wherein said alarm state comprises 
automatic opening of a video link to a central controller. 


1 8. The device of claim 13, said alarm state being additionally 
triggerable by at least one of an instabiUty monitor, and a loud sound monitor. 

1 9. The device of claim 1 , sized and configured for mounting 
unobtrusively on a subject. 

20. The device of claim 1, further comprising location detection 
functionality for determining a location, said apparatus further being 
configured to report said location. 

21. The device of claim 20, wherein said location detection 
functionality is one of a group comprising a GPS detector and a triangulation 
system. '' • - - - .-v . -. . 

22. The device of claim 1, further comprising a direction sensor, said 
direction sensor comprising a compass needle and functionality for measuring 
an angle of said compass needle in relation to a reference. 

23. The device of claim 1, associated with a memory stack for storing 
a predetennined amount of immediately preceding data, said detector being 
configured to save all data in said stack upon entry into said alarm state. 

24. The device of claim 1, further configured to apply scoring to an 
action of said subject. 

25. The device of claim 24, wherein said action is scored in relation to a 
feature in virtual reality. 
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26. The device of claim 24, wherein said action is scored in relation 
to a predetermined program. 


27. The device of claun 23, wherein said predetermined amount of 
data is usable for animatmg said figure. 

28. Device for virtual reality representation of a subject being 
monitored, the apparatus comprising: 

a measurement part comprising a motion unit for receiving body motion 
data from said subject, and 

a monitoring unit for using signals received from said measurement part to 
animate a virtual animation character at a remote location, thereby to provide 
an indication of said overall state. 

29. The device of claim 28, wherein said motion unit comprises 
passive position detectors connected to a pfuraUty of to&tions on sai^ subject 
and a triangulation unit for detecting relative motions of said passive position 
detectors. . 

30. The device of claim 28, wherein said motion tmit further 
comprises an inclination detector affixed to said subject. 

3 1 . The device of claim 28, wherein said motion unit comprises an 
accelerometer. 

32. The device of claim 28, wherein said measurement unit is 
responsive to transmitter units placed on said subjects. 

33. The device of claim 28, wherein said measurement unit is 
operable to enter an alarm state under predetermined conditions and said alarm 
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state comprises automatic opening of a communication channel to a central 
controller. 

34. The device of claim 28, wherein said predetermined conditions 
include a position likely to cause injury. 

35. The device of claim 33, wherein said alarm state comprises 
automatic opening of an audio channel to a central controller. 

36. The device of claim 35, wherein an end of said audio channel is 
located on said subject. 

37. The device of claim 33, wherein said alarm state comprises 
automatic opening of a video link to a central controller. 

38. The device of claim 33, said alarm state being additionally 
triggerable by at least one of an instability indhitbr; aasVloua^soUhd monitor. 

39. The device of claim 28, sized and configured for mounting 
unobtrusively on, a subject. 

40. The device of claim 28, further comprising location detection 
functionality for determining a location, said apparatus further being 
configured to report said location. 

41. The device of claim 40, wherein said location detection 
functionality is one of a group comprising a GPS detector and a triangulation 
system. 
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42. The device of claim 28, further comprising a direction sensor, 
said direction sensor comprising a compass needle and functionality for 
measuring an angle of said compass needle in relation to a reference. 

43 . The device of claim 28, associated with a memory stack for 
storing a predetermined amoxmt of immediately preceding data, said detector 
being configtired to save all data in said stack upon entry into said alarm state. 

44. The device of claim 28, further configured to apply scoring to an 
action of said subject. 

45. The device of claim 44, wherein said action is scored in relation to a 
feature in virtual reality, 

46. The device of claim 44, wherein said action is scored in relation 
to a predetermined program. 

47. The device of claim 43, wherein said predetennined amount of 
data is usable for animating said figure. 

48. Device for virtual reality representation of a subject being 
monitored, substantially as hereinbefore described with reference to the 
accompanying drawings. 

49. Virtual reality representation method of a subject being 
monitored, substantially as hereinbefore described with reference to the 
accompanying drawings. 


David Cohen 
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